Search for: All records

Training large language models (LLMs) increasingly relies on geographically distributed accelerators, causing prohibitive communication costs across regions and uneven utilization of heterogeneous hardware. We propose HALoS, a hierarchical asynchronous optimization framework that tackles these issues by introducing local parameter servers (LPSs) within each region and a global parameter server (GPS) that merges updates across regions. This hierarchical design minimizes expensive inter-region communication, reduces straggler effects, and leverages fast intra-region links. We provide a rigorous convergence analysis for HALoS under non-convex objectives, including theoretical guarantees on the role of hierarchical momentum in asynchronous training. Empirically, HALoS attains up to 7.5x faster convergence than synchronous baselines in geo-distributed LLM training and improves upon existing asynchronous methods by up to 2.1x. Crucially, HALoS preserves the model quality of fully synchronous SGD-matching or exceeding accuracy on standard language modeling and downstream benchmarks-while substantially lowering total training time. These results demonstrate that hierarchical, server-side update accumulation and global model merging are powerful tools for scalable, efficient training of new-era LLMs in heterogeneous, geo-distributed environments. 

Multiple silicon solar cell technologies have surpassed or are close to surpassing 26% efficiency. Dielectric and amorphous silicon-based passivation layers combined with minimal metal/silicon contact areas were responsible for reducing the surface saturation current density below 3 fA cm −2 . At open-circuit, in passivated contact solar cells, the recombination is mainly from fundamental mechanisms (Auger and radiative) representing over 3/4 of the total recombination. At the maximum power point, the fundamental recombination fraction can drop to half, as surface and bulk Shockley–Read–Hall step in. As a result, to further increase the performance at the operating point, it is paramount to reduce the bulk dependence and secure proper surface passivation. Bulk recombination can be mitigated either by reducing bulk defect density or by reducing the wafer thickness. We demonstrate that for commercially-viable solar-grade silicon, thinner wafers and surface saturation current densities below 1 fA cm −2 , are required to significantly increase the practical efficiency limit of solar cells up to 0.6% absolute. For a high-quality n-type bulk silicon minority-carrier lifetime of 10 ms, the optimum wafer thickness range is 40–60 μm, a very different value from 110 μm previously calculated assuming undoped substrates and solely Auger and radiative recombination. In this thickness range surface saturation current densities near 0.1 fA cm −2 are required to narrow the gap towards the fundamental efficiency limit. We experimentally demonstrate surface saturation currents below 0.5 fA cm −2 on pi/CZ/in structures across different wafer thicknesses (35–170 μm), with potential to reach open-circuit voltages close to 770 mV and bandgap-voltage offsets near 350 mV. Finally, we use the bandgap-voltage offset as a metric to compare the quality of champion experimental solar cells in the literature, for the most commercially-relevant photovoltaic cell absorbers and architectures.